Scalable Sparse Covariance Estimation via Self-Concordance

We consider the class of convex minimization problems, composed of a self-concordant function, such as the log det metric, a convex data fidelity term h(·) and, a regularizing – possibly non-smooth – function g(·). This type of problems have recently attracted a great deal of interest, mainly due to their omnipresence in top-notch applications. Under this locally Lipschitz continuous gradient setting, we analyze the convergence behavior of proximal Newton schemes with the added twist of a probable presence of inexact evaluations. We prove attractive convergence rate guarantees and enhance state-of-the-art optimization schemes to accommodate such developments. Experimental results on sparse covariance estimation show the merits of our algorithm, both in terms of recovery efficiency and complexity. Introduction Convex `1-regularized log det divergence criteria have been proven to produce – both theoretically and empirically – consistent modeling in diverse top-notch applications. The literature on the setup and utilization of such criteria is expanding with applications in Gaussian graphical learning (Dahl, Vandenberghe, and Roychowdhury 2008; Banerjee, El Ghaoui, and d’Aspremont 2008; Hsieh et al. 2011), sparse covariance estimation (Rothman 2012), Poissonbased imaging (Harmany, Marcia, and Willett 2012), etc. In this paper, we focus on the sparse covariance estimation problem. Particularly, let {xj}Nj=1 be a collection of nvariate random vectors, i.e., xj ∈ R, drawn from a joint probability distribution with covariance matrix Σ. In this context, assume there may exist unknown marginal independences among the variables to discover; we note that (Σ)kl = 0 when the k-th and l-th variables are independent. Here, we assume Σ is unknown and sparse, i.e., only a small number of entries are nonzero. Our goal is to recover the nonzero pattern of Σ, as well as compute a good approximation, from a (possibly) limited sample corpus. ∗This work is supported in part by the European Commission under Grant MIRG-268398, ERC Future Proof and SNF 200021132548, SNF 200021-146750 and SNF CRSII2-147633. Copyright c © 2014, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Mathematically, one way to approximate Σ is by solving: